In this paper we consider the problem of computing an $\epsilon$-optimal policy of a discounted Markov Decision Process (DMDP) provided we can only access its transition function through a generative sampling model that given any state-action pair samples from the transition function in $O(1)$ time.

This work tackles the problem of robust planning in non-stationary stochastic environments.

In this work, we study the relationship between these objectives.

We give a model-based agent that builds a Python program representing its knowledge of the world based on its interactions with the environment.

Most reinforcement learning (RL) algorithms hinge on the Markovian assumption, i.e. that the underlying system transitions and rewards are Markovian in some natural notion of (observable)